Software Testing Overview

Principles

Definition

Software testing is a systematic process and academic discipline concerned with the empirical investigation of software behavior to identify discrepancies between expected and actual outcomes, evaluate quality characteristics, and verify conformance to specified requirements. It encompasses methodologies, techniques, and tools aimed at enhancing software reliability and validating system functionality.

Ontological type

Core Techniques

Testing Levels

Test Automation

Key developments

Key Figures

Reliability and Testability era

Jakob Nielsen [1] was associated with the Technical University of Denmark [3] and the University of Copenhagen [4] during this era. His key contribution in this era centers on Usability inspection methods [6], which formalized early evaluation practices to improve software reliability and testability. Donald Musa [2] was affiliated with the University of Pittsburgh [5] during this era. Musa's key contributions in this era include A theory of software reliability and its application [8], Software reliability measurement [7], and Validity of Execution-Time Theory of Software Reliability [9], which collectively advanced quantitative reliability modeling and measurement-driven assessment.

Empirical Reproducible Testing era

Gregg Rothermel [1] is a leading figure in empirical software testing during 2000–2017, with institutional involvement at University of California, Irvine [3] and Korea Advanced Institute of Science and Technology [4]. His key contributions include empirical evaluation of test-case prioritization, as demonstrated by Test case prioritization: a family of empirical studies [7], and an empirical study of the effects of minimization on the fault detection capabilities of test suites [8], which informed resource allocation and repeatable evaluation in this era. Michael D. Ernst [2] is a prominent researcher in this era, with affiliations at Harvard University [5] and Massachusetts Institute of Technology [6]. Defects4J: a database of existing faults to enable controlled testing studies for Java programs [9] stands as Ernst's notable contribution to this era, providing a standardized benchmark enabling reproducible experimentation and robust evaluation of testing techniques.

Predictive ML-driven Testing era

Michael Hicks [1] is associated with University of Maryland, College Park [3] and New York University Press [4] during the Predictive ML-driven Testing era. His key contribution in this era centers on evaluating fuzz testing as a validation mechanism for prediction-driven testing strategies, as demonstrated by the 2018 paper Evaluating Fuzz Testing [7], underscoring its role in guiding risk-based test prioritization. Lu Zhang [2] has associations with Center for Life Sciences [5] and Peking University [6] during this era. Her key contribution includes identifying patch correctness in test-based program repair [8], illustrating how patch correctness assessment informs predictive testing and repair quality under prioritized regimes.